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This study investigates the effectiveness of the online Flash game Disease Defenders in producing knowledge 
gains for concepts related to the scientific process. Disease Defenders was specifically designed to model how 
the scientific process is central to a variety of disciplines and science careers. An additional question relates 
to the game's ability to shift attitudes toward science. Middle school classes from grades six to eight were 
assigned to the experimental group (n = 489) or control group (n = 367) and asked to participate in a three- 
session intervention. The sessions involved completing a pretest, a game play session, and taking a posttest. 
Students in the experimental group played Disease Defenders while students in the control group played an 
alternative science game. Results showed a significant increase in mean science knowledge scores for all 
grades in the experimental group, with sixth grade and seventh grade students gaining more knowledge 
than eighth grade students. Additionally, results showed a significant positive change in science attitudes 
only among sixth graders, who also rated their satisfaction with the game more favorably than students in 
higher grades. No differences in mean test scores were found between genders for science knowledge or 
science attitudes, suggesting that the game is equally effective for males and females. 



INTRODUCTION 

Teachers at all educational levels use textbooks to sup- 
port science instruction, but studies have revealed the inad- 
equacy of textbooks in representing the scientific process. A 
recent study published in the Journal of Microbiology & Biology 
Education determined that introductory undergraduate 
biology textbooks under-represent the scientific process, 
which could be an impediment to students' understanding of 
science (9). Similarly, the K-12 classroom oftentimes covers 
the scientific process and required skills in isolation from 
content at the beginning of the school year with textbooks 
reinforcing this separation (26). This is problematic because 
the teaching of science should mirror the processes used 
by real scientists, which are not separated from content 
and involve hands-on investigations. How are students at all 
educational levels to succeed in understanding the scientific 
process when they have resources that impede their ability 
to do so? These findings highlight the need for innovative 
materials that provide students a stronger conceptual depic- 
tion of the scientific process. 
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Digital games can enable students to experience the 
scientific process by providing virtual laboratory or field 
investigations that overcome constraints that teachers face, 
such as lack of laboratory facilities, laboratory supplies, 
or long distances from science museums (24). Computer- 
based science games have previously been demonstrated to 
increase learning in various science content areas (2, 14, 15, 
19-22, 27), suggesting a place for games in supplementing 
science instruction. In addition, computer-based science 
instruction (28) and a narrative-based online game (14) have 
been shown to positively influence attitudes toward science, 
suggesting that digital games could serve as a teaching tool 
for mitigating the negative shifts in attitudes toward science 
observed in middle school and high school students (II, 12). 

Given the gender gaps that have been reported in sci- 
ence, technology, engineering, and mathematics (STEM) 
areas (25), developers of any school-based science educa- 
tion program have to be sensitive to potential gender dif- 
ferences in the program's efficacy, likeability, and usability. 
Digital games in particular were, for a longtime, considered 
a male leisure activity. However, recent statistics refute 
major gender differences regarding the playing of video 
games: 99% of boys vs. 94% of girls reported playing video 
games in a recent Pew Report (17), and researchers have 
found that both genders benefit equally from science-based 
digital games (I, 7, 13). 

Our previous research has demonstrated learning gains 
in microbiology content among middle and high school 
students who used MedMyst, an online Flash-based Web 
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adventure series containing virtual lab experiments and 
arcade-style games set within an engaging mystery story 
line (http://medmyst.rice.edu/) (20). However, the focus 
of the three MedMyst missions previously evaluated is on 
teaching microbiology content and science history, not 
the scientific process. Also, the prior evaluation did not 
investigate the ability of MedMyst to influence students' 
attitudes toward science. 

To explore the potential of an online game to teach the 
scientific process within the context of an engaging microbi- 
ology scenario, the current MedMyst Web adventure series 
was expanded to include an additional mission called Disease 
Defenders. The objective of Disease Defenders is to illustrate 
the scientific process, including constructing hypotheses, 
defining variables, defining experimental and control groups, 
testing hypotheses, analyzing data, and drawing conclusions. 
In addition, Disease Defenders portrays the scientific process 
as utilized by three different infectious disease specialists: a 
microbiologist, epidemiologist, and veterinarian. The pur- 
pose of including three different infectious disease specialists 
in Disease Defenders is to show students that there is not a 
single "scientific method" used by scientists, which is con- 
sistent with the goals of the conceptual framework for the 
new K— 12 science education standards (7). We hypothesized 
Disease Defenders to be effective in producing knowledge 
gains for concepts related to the scientific process among 
middle school students and we hypothesized middle school 
students would report more positive attitudes toward sci- 
ence after exposure to the game. 

METHODS 

Intervention 

With input from middle school science teachers, we 
created several learning objectives that align to the National 
Science Education Standards (23), which were then used in 
the game design process to guide the development of the 
storyline and game activities (Table I). An advisory panel of 
infectious disease experts reviewed the game content for 
scientific accuracy. 



Disease Defenders begins with a short dialogue explain- 
ing that investigating disease outbreaks requires a team of 
experts. Players are then asked to choose, in any order, 
three different career "paths" (veterinarian, epidemiologist, 
and microbiologist), each with a virtual infectious diseases 
specialist portrayed as a "disease defender." Guided by the 
on-screen scientists, in the veterinarian path, players per- 
form a diagnostic procedure; in the epidemiology path, play- 
ers conduct a case-control study; and in the microbiology 
path, players perform a traditional laboratory experiment 
(Fig. I). By involving students in the process of conducting a 
scientific investigation from three different career perspec- 
tives, they experience how the scientific process is applied 
in different contexts. All of the components of the scientific 
process are included in each career path; however, each path 
emphasizes some components more than others (Table I). 
For example, the microbiology career path places emphasis 
on hypothesis development and identification of independent 
and dependent variables; however, students are still exposed 
to hypothesis testing, data analysis, and drawing conclusions 
from the results of an experiment. 

In addition to the scientific process, rabies is another 
common thread that runs through each path. However, the 
focus is not on teaching about the rabies pathogen. Rather, 
showing how different scientists apply the scientific process 
to investigate different aspects of the same infectious disease 
is used to further facilitate students' contextual understand- 
ing of scientific investigations. Based on the results of prior 
research in which student engagement was positively cor- 
related with learning outcomes, another reason for choos- 
ing a common storyline was to maximize the potential for 
student interest and engagement (27). Placing the content 
within a storyline also situates the learning in a context 
that necessitates problem solving and provides structure 
to the educational content, both of which have previously 
been established as important factors in producing effective 
educational games (3, 5, 10, 29, 30). 

Another technique used to support learning is allow- 
ing players to interact with virtual scientists, who provide 
instructions on what to do next, give feedback, and check 
for understanding, as illustrated in Figure 2. 



TABLE I. 

Learning objectives, in-game activities, and science knowledge questions for Disease Defenders. 



Learning Objective 

Formulate a hypothesis and identify inde- 
pendent and dependent variables. 

Use experimental and control groups to 
draw conclusions from an experiment. 

Interpret results and draw conclusions 
about a case-control study. 



In-Game Activity 

Microbiology Path: Perform a simple filtering experiment to attempt 
to remove the rabies pathogen from a liquid. 

Veterinarian Path: Examine brain samples using microscopy to 
determine if a pig died from rabies. 

Epidemiology Path: Conduct a case-control study to determine 
the source of a rabies outbreak. 



Questions 3 

1,4, 10, 13, 14, I6 b , 17 

6 b ,7, I I, I5 b 
2,3 b ,5,8,9, 12 



a Science knowledge questions can be found in Appendix I. 
indicates general science knowledge questions. 
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FIGURE I. From top to bottom, screen captures from the begin- 
ning of the veterinarian, epidemiologist, and microbiologist paths 
in Disease Defenders showing the problems players are asked to 
investigate. 



Participants 

Approval to conduct the study was obtained from Rice 
University's Institutional Review Board. An online solicita- 
tion was sent out to middle school science teachers who 
had attended various MedMyst professional development 
workshops. An incentive of $ 1 50 for organizing the field test 
was offered to teachers. To participate in the field test, sci- 
ence teachers were required to meet the following inclusion 
criteria: permission from the school principal, access to a 
computer lab for three class periods, guarantee to provide 
at least 50 students, and collection of informed consent 
forms signed by both parents and students. 

The final sample included 17 teachers from 15 differ- 
ent public schools across 1 1 US states. This comprised 856 
students (51% female) with 98 of the students in sixth grade 
(11.4%), 461 in seventh grade (53.9%), and 297 in eighth 




FIGURE 2. Screen capture of a check for understanding during 
the veterinarian path of Disease Defenders. Players are asked to 
identify and add experimental, positive, and negative control 
brain samples to microscope slides for staining to determine if a 
pig died from rabies. 



grade (34.7%). Of the students involved in the study, 72% 
identified themselves as Caucasian, 12.7% Hispanic/Latino, 
8.1% Asian/Pacific Islander, 2.9% African-American, 0.9% 
Native American, and 3.4% Other. To validate the repre- 
sentation of socio-economic status diversity, students in the 
sample were assigned to one of three economic groups as 
operationalized by free/reduced lunch percentages of the 
total school population, ranging from low disadvantage (less 
than 29% of students receive free/reduced lunch), to middle 
disadvantage (29-59% of students receive free/reduced 
lunch), and high disadvantage (more than 59% of students 
receive free/reduced lunch). The final sample contained 
19.1% (n = 163) low-disadvantage students, 50.1% (n = 429) 
middle-disadvantage students, and 30.8% (n = 264) high- 
disadvantage students. 

Procedure 

The 17 participating teachers were assigned to either 
the experimental group or control group based on the grade 
level they teach in order to ensure equal representation 
of sixth, seventh, and eighth graders between groups. All 
students who provided parental consent participated in 
three sessions in the school computer lab. During Session 
One (approximately 30 minutes), students in both groups 
were asked to provide their assent, were asked for an 
identification number assigned by the teacher so students' 
identities could not be matched to data, were asked basic 
demographic information (e.g., grade, gender, ethnicity), 
and responded to 17 multiple-choice knowledge questions 
and science-related attitudinal scales. During Session Two, 
scheduled a minimum of three days after Session One, 
students in the experimental group played Disease Defend- 
ers (approximately 30 minutes total). Students assigned to 
the control group played a different science game from 
the MedMyst series, which is comparable in length, but 
does not focus on the scientific process. Use of a control 
group in pretest/posttest designs permits understanding 
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whether effects of learning are a function of the intervention 
or threats to internal validity (i.e., testing effects, history, 
maturation, or game-play processes). After an additional 
three-day delay, students in both groups completed the 
posttest in Session Three (approximately 30 minutes), 
which contained the same knowledge test and the same 
science-related attitudinal scales that were in the pretest. 
In addition, participants were asked to rate the usability and 
their satisfaction with the game. The minimum three-day 
time delay between game play and posttesting was purposely 
planned to test the persistent effects of the game beyond 
the immediacy of the intervention. 

Measures 

Science knowledge. Development of the science 
knowledge scale consisted of infectious disease experts and 
middle school science teachers reviewing 25 pilot questions 
to ensure reflection of game content, scientific accuracy, and 
appropriateness for middle school. Questions were derived 
from the scientific process, content, and careers presented 
within the three career paths in Disease Defenders (Table I, 
Appendix I) and consisted of recall and application ques- 
tions. All items were multiple-choice with three or four 
response options. Item-by-item analysis of pilot data from 
students not participating in the field test refined the scale 
down to 17 items. Within the 17 items there were two 
different question types: 13 related to scientific process 
knowledge and four related to general science knowledge. 
The internal consistency reliability (Cronbach's a) was 0.63 
for the science knowledge pretest and 0.79 for the posttest. 

Attitudes. The attitude questions included three 
scales: attitudes toward science, satisfaction with the game, 
and game usability (see Appendix I). All three scales have 
been previously used and validated (14, 15, 19). For each 
scale, participants read statements and rated their level of 
agreement on a five-point Likert scale from I = Strongly 
Disagree to 5 = Strongly Agree. The internal consistency 
reliability (Cronbach's a) was 0.75 for both the pretest and 
posttest attitudes toward science scale, and 0.88 and 0.61 for 
satisfaction and usability scales, respectively (posttest only). 

RESULTS 

Of the original 1231 middle school students who con- 
sented, 375 were excluded from the analyses for the follow- 
ing reasons: identification numbers could not be matched 
from pretest to posttest, students left answers blank, or 
students reported not finishing the game. The final sample 
included 856 students: 489 in the experimental group and 
367 in the control group. 

Science knowledge acquisition 

To assess science knowledge acquisition, mean test 
scores in both the experimental and control groups for the 



17 science knowledge questions were calculated (Table 2). 
For the experimental group, the mean test score increased 
from 7.49 (44.1%) before playing Disease Defenders to 9.60 
(56.5%) after playing the game. In contrast, the group mean 
test score in the control group slightly decreased from 6.68 
before playing the control game to 6.35 after playing. How- 
ever, the comparison of pretest scores (7.49 for experimen- 
tal and 6.68 for the control) using an independent-samples 
t-test revealed that the scores were significantly different 
(t[854] =4.16, p< 0.01). 

Given this difference, a General Linear Model (GLM) 
was used to compute posttest scores adjusted for these 
differences. Posttest scores were predicted by group, 
pretest scores, and the interaction of group and pretest 
scores, which takes into account the different slopes for the 
two groups. The pretest scores were centered (converted 
to deviation-scores) before the cross-product term was 
computed so that the effect of groups would represent the 
average effect of groups. 

The resulting adjusted posttest scores are shown in 
Table 2; the experimental group mean score was adjusted 
from 9.60 to 9.30 and the control group mean score was 
adjusted from 6.35 to 6.58. The difference between the 
adjusted posttest means for the experimental and control 
group was statistically significant, F(l,852) = 223.70, p < 0.001 
and the effect size was large d = 1.05 (6), indicating that 
students in the experimental group who played Disease 
Defenders learned significantly more than students in the 
control group who did not play the game. 

Attitudes measures 

To assess shifts in attitudes toward science, mean test 
scores in both the experimental and control groups for the 
four science attitudes questions were calculated. For the 
experimental group, the science attitudes mean score shifted 
only slightly in a positive direction from 3.98 (SD = 0.75) to 
4.01 (SD = 0.76) (with 5.0 being most positive) after play- 
ing the game, and a paired-samples t-test revealed that the 
shift was not significant (t[488] = 1.21, p = 0.227). For the 
control group, the mean score shifted slightly in a negative 
direction from 3.99 (SD = 0.71) to 3.92 (SD = 0.77) after 
playing. A paired-samples t-test revealed that the negative 
shift was significant (t[366]=2.59, p < 0.05); however, the 
effect size was small d = 0.18. 

Usability and satisfaction with the Disease Defenders 
intervention were measured posttest only in the experi- 
mental group with usability posttest group means at 3.75 
(SD = 0.70) and satisfaction posttest group means at 3.93 
(SD = 0.80), with 5.0 being the most positive for both scales. 

The science knowledge acquisition and attitudes mea- 
sures analyses capture the overall effects of the Disease 
Defenders intervention compared to a control group. Below 
we provide a more extensive analysis of the Disease Defend- 
ers intervention by examining differences by question type, 
grade level, and gender within the experimental group only. 
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TABLE 2. 

Pretest, posttest, and adjusted posttest means for overall science knowledge questions for the experimental and control groups be- 
fore and after playing the assigned MedMyst game. 



Group (n) 


Pretest Mean (SD) 


Posttest Mean (SD) 


Adjusted Posttest Mean (SD) 


Experimental (489) 


7.49 (3.10) 


9.60 (3.95) 


9.30 (2.92) 


Control (367) 


6.68 (2.40) 


6.35 (2.40) 


6.58 (2.08) 



Science knowledge acquisition by question type 

To assess science knowledge acquisition by question 
type, mean scores for general science knowledge questions 
and scientific process knowledge questions were calculated 
separately. For general science knowledge questions, the 
mean score increased from 2.12 (SD = 1.13) to 2.32 (SD 
= 1.21) after playing the game, and a paired-samples t-test 
revealed the shift was significant (t[488] = 3.54, p < 0.01); 
however, the effect size was small d = 0.23. A second 
paired-samples t-test revealed a significant positive shift 
(t[488] = 18.70, p < 0.01) in mean scores for scientific 
process knowledge questions from 5.37 (SD = 2.50) to 
7.43 (SD = 3.20) and a large effect size (d = 1.23), indicating 
that learning gains for the overall science knowledge scale 
were driven by the scientific process knowledge items. 

Differences between genders 

A repeated measures ANOVA was conducted in GLM 
to examine gender differences in science knowledge acqui- 
sition. Pretest and posttest were entered as the repeated 
measures and gender was entered as a between-subject 
variable. There was no interaction between gender and 
learning (F(l,487) = 3.51, p = 0.062), suggesting that Disease 
Defenders was equally effective for both boys and girls. 

To examine changes in science attitudes as a func- 
tion of gender, a similar analysis in GLM was conducted 
with pre- and posttest science attitudes entered as the 
repeated measures and gender as a between-subject 
variable. This analysis revealed no difference in attitudes 
toward science between genders (F(l ,487) = 0.338, p = 
0.561). Furthermore, an independent-samples t-test with 
female and male students revealed that neither satisfac- 
tion nor game usability ratings differed between genders 
(satisfaction: t(443) = -0.14, p = 0.07; game usability: 
t(477) = -0.102, p = 0.77). 

Differences across grades 

Grade-level differences in the efficacy, likability, and 
usability of Disease Defenders were also examined in the 
experimental group. Although differences in pretest science 
knowledge were expected, with higher grades having more 
baseline knowledge than lower grades, we expected that 
each grade would show an increase in science knowledge. 



To examine the effects of knowledge, a repeated measures 
ANOVA was conducted in GLM with pretest and posttest 
entered as the repeated measures and grade as a between- 
subject variable. The amount of learning was significantly 
different across grades (F(2,486) = 8.08, p < 0.01) (Fig. 3). 
Although paired-samples t-tests performed separately for 
each grade revealed that the increase in science knowl- 
edge scores between pretest and posttest is significant in 
all grades, the comparison of group means showed that 
knowledge gains were higher among sixth grade students 
(increased 16.9% from 6.02 to 8.90, t(49) = -7.7, p < 0.01; 
n = 50) and seventh grade students (increased 14.9% from 
7.76 to 10.30, t(299) = 15.46, p < 0.01; n = 300) compared 
to eighth grade students (increased 8.5% from 7.42 to 8.87, 
t(l38) = -5.57, p< 0.01; n= 139). 

A similar analysis in GLM was performed to examine 
science attitudes with pre- and posttest attitude assess- 
ments as the repeated measures and grade as a between- 
subject variable. The results showed a significant difference 
in attitudes toward science between grades (F(2,486) = 
6.58, p < 0.01) (Fig. 4). Separate paired-samples t-tests 
revealed that the difference in attitudes is driven by a 
significant positive change of science attitude between pre- 
and posttest among sixth grade students (t(49) = -3.03, p 
< 0.01), whereas both seventh and eighth grade students' 
science attitudes did not change significantly (grade seven: 
t(299) = 0.086, p = 0.921; grade eight: t(l38) = -0.1 1 1, 
p = 0.912). 

Grade-level differences in satisfaction were examined 
with an ANOVA. Results showed a significant difference 
(F(2,486) = 6.22, p < 0.01) in satisfaction such that students 
in lower grades rated the game more positively than those 
in higher grades (sixth grade mean: 4.09 (SD 0.76), seventh 
grade mean: 3.99 (SD 0.77), eighth grade mean 3.73 (SD 
0.85) on a five-point Likert scale with 5 being the most 
positive). There was no difference in usability ratings across 
grades (F(2,486) = 0.508, p = 0.602). 

In summary, students in grades six and seven gained 
more knowledge after playing the game than eighth grade 
students, yet learning gains were significant in all grades. 
Moreover, although the shift in attitudes toward science in 
the experimental group as a whole was not significant, the 
analysis across grades revealed a difference, with a significant 
positive change in attitudes only among sixth graders, who 
also rated their satisfaction with the game more favorably 
than students in higher grades. 
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FIGURE 3. Interaction between science knowledge scores and 
grade. Error bars represent standard errors. Grade six pretest: 
mean 6.02 (SD 2.07), posttest: mean 8.90 (SD 2.78). Grade seven 
pretest: mean 7.76 (SD 3.04), posttest: mean 10.30 (SD 3.93). 
Grade eight pretest: mean 7.42 (SD 3.38), posttest: mean 8.87 
(SD4.30). 



FIGURE 4. Interaction between attitudes toward science and 
grade. I = Most Negative and 5 = Most Positive attitude toward 
science. Error bars represent standard errors. Grade six pretest: 
mean 3.93 (SD 0.96), posttest: mean 4.23 (SD 0.76). Grade seven 
pretest: mean 4.04 (SD 0.7I), posttest: mean 4.04 (SD 0.75). Grade 
eight pretest: mean 3.87 (SD 0.75), posttest: mean 3.87 (SD 0.75). 



DISCUSSION 

The results of this evaluation support the hypothesis 
that Disease Defenders is effective at producing scientific 
process knowledge gains among middle school students. 
The demonstrated learning gains, resulting from students 
interacting with the game for only one class period, have 
a large effect size and are in line with results from previ- 
ous evaluations of similar computer-based games (2, 14, 
1 5, 19-22, 27). Although the average posttest score in the 
experimental group was only 56.5%, perhaps combining the 
game with other instructional interventions would produce 
even higher knowledge gains. For this study, teachers were 
specifically asked to refrain from reinforcing or discussing 
any of the game's content until after the posttest. Normally, 
one would expect teachers to use games such as Disease 
Defenders as part of a learning cycle. Often teachers use 
the games as part of the Engage or Explore phase in the 5E 
Model (4); however, the experimental design was structured 
to test the efficacy of the intervention without the benefit 
of classroom reinforcement. Future evaluations including 
control groups exposed to standard classroom instruction 
on the scientific process compared to experimental groups 
incorporating Disease Defenders plus standard classroom 
instruction would further elucidate the contribution of the 
game in teaching the scientific process. 

Digital games could provide a way to help shape at- 
titudes toward science. This variable is important because 
during middle school and high school, science attitudes tend 
to shift negatively among students (II, 1 2). Studies also show 
that students who in eighth grade indicate interest in sci- 
ence and science careers are more likely to enroll in science 
courses and earn STEM degrees (18, 31), so middle school is a 
good age to target for shifting science attitudes and fostering 
interest in science. This is particularly important in light of 
the concern that the United Sates is not preparing enough 



students in the areas of STEM (1 6). In contrast to positive 
learning gains, the analysis of attitudes toward science among 
the experimental groups as a whole did not reveal a positive 
shift. These results could be due to multiple reasons, the 
first being the relatively high initial predisposition toward 
science reported by students, which makes it difficult to 
shift attitudes even higher. Students reported mean pretest 
science attitudes scores at 3.98, which is higher than the 
3.0 Likert scale mid-point (5.0 being the most positive), 
and, after playing the game, reported mean posttest scores 
only slightly higher at 4.0I. Secondly, although the science 
attitudes scale has a high reliability and was validated in 
prior studies (14, 15, 19), it consisted of only four questions 
and may not have been sensitive enough to capture small 
changes in attitude. Finally, while other studies demonstrated 
positive attitude shifts after exposure to computer-based 
science instruction (14, 28), those interventions consisted 
of longer exposures. Perhaps the length of Disease Defend- 
ers (approximately 30 minutes) was too short to influence 
students' attitudes toward science. 

A surprising finding was a small negative shift in students' 
science attitudes in the control group. A possible explanation 
could be that students in the control played a different sci- 
ence game from the MedMyst series that does not focus on 
the scientific process, yet the pretest and posttest questions 
focused on scientific process. It is possible that students 
may have become frustrated answering questions that had 
nothing to do with the content presented in the game they 
played. To prevent frustration, future study designs could 
expose the control group to the same intervention as the 
experimental group after the posttest instead of between 
pre- and posttest. 

Even though statistics now show that males and females 
both play digital games equally (17), the gender gaps that have 
been reported in STEM areas (25) mandate the analysis of 
possible gender differences in the efficacy of digital games. No 
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statistically significant differences were found in mean scores 
for attitudes, satisfaction, usability, and scientific process 
knowledge, indicating that Disease Defenders does not have 
a gender bias and is equally effective for males and females 
in middle school. These results are in line with the finding 
that both genders benefit from science-based digital games 
and multi-user virtual environments (I, 8, 13, 14, 20, 21). 

Since scientific practices are reinforced in sixth, seventh, 
and eighth grades (23), Disease Defenders was designed to 
be equally beneficial to students across all middle school 
grades. However, the analysis of grade differences revealed 
that students in sixth grade and seventh grade learned more 
from the game than students in eighth grade. In addition to 
the difference in science knowledge acquisition, the interven- 
tion resulted in a significant positive shift of science attitudes 
among sixth grade students, while such a shift was not found 
in the higher grades. Together with the fact that sixth graders 
also rated their satisfaction with the game higher than older 
students, these findings suggest that Disease Defenders may 
be more efficacious and appealing in sixth and seventh grade. 
It is possible that eighth graders' low satisfaction with the 
game may have influenced their attitudes. However, due to 
the widely varying sample sizes (sixth grade n = 50, seventh 
grade n = 300, and eighth grade n = 1 39) these results should 
be taken with caution. Further research is necessary to de- 
termine what makes games appealing to different age groups. 

While the evaluation of Disease Defenders demonstrated 
grade-level differences, students at all levels in middle school 
demonstrated science knowledge gains. Unsolicited feed- 
back indicates the broad appeal of MedMyst across many 
grades, including high school and college. With the number 
of textbooks under-representing the scientific process, Dis- 
ease Defenders could provide an interactive tool for teachers 
to use as a means to reinforce the scientific process and 
foster scientific thinking. 

SUPPLEMENTAL MATERIALS 

Appendix I: Pre- and posttest items 
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